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Abstract 

This paper describes two models of assessment 
of undergraduate business learning in two similar 
universities located on the East Coast of the United States. 
Both models stem from seeking continued accreditation 
by the AACSB and are focused on a group of student skills 
identified by the faculty as essential to undergraduate 
business education. The assessment processes within 
the two schools differ in their approaches to methods 
of measurement. One school used a top-down model, 
which was created by administrators and operated 
outside the normal coursework to assess students in the 
program. The other school used methods embedded in 
the courses to only assess programs (i.e., a bottom-up 
model). Whereas the learning goals, course structures, 
and faculty composition were similar for the two 
schools, the bottom-up approach involved more faculty 
members in the assessment process. The article details 
the measures and rubrics employed by each school, 
the merits and shortcomings of each approach, and a 
recommended assessment model combining the best 
elements of the two paradigms. Currently, the results 
of the two assessment processes indicate that both 
schools must improve the feedback loop for continuous 
improvement. 
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Top-Down Versus Bottom-Up 
Paradigms of Undergraduate 
Business School Assurance of 
Learning Techniques 

Introduction 

In 1 993, in reference to higher education, 

Astin wrote that "practically everybody in the 
academic community gets assessed these days, and 
practically everybody assesses somebody else" (p. 

1 ). However, a full 1 8 years after Astin wrote these 
words, many business schools are still struggling 
with designing and implementing assessment 
programs. In higher education, assessment is 
defined as systematically collecting, reviewing, 
and using information about the learning and 
educational performance of students enrolled 
in academic programs in order to improve the 
students' learning (Palomba & Banta, 1999). The 
process consists of outlining goals and objectives 
for learning, determining the most appropriate 
samples and methods to examine, collecting 
educational data, and using the information to 
improve programs. 

Business schools have been slow to adopt 
assessment policies and procedures because of 
the difficulty of determining what, when, how, and 
whom to assess. Academic culture often hinders 
assessment because tenured faculty may not be 
responsive to administrators who attempt to alter 
course content for the purposes of assessment 
and continuous improvement. Some faculty 
members believe that they are already assessing 
student learning in their classrooms and additional 
measures are wasteful. However, the accreditation 
boards have embraced assessment, and schools 
must show efforts to assess student outcomes for 
reaccreditation. 

Planning and implementing an assessment 
program in higher education is generally the result 
of coming up for accreditation or reaccreditation 
by an external, reputable professional organization. 
For university-level business programs, the 
primary accrediting board is the Association to 
Advance Collegiate Schools of Business (AACSB 



International), whose mission is to "advance quality 
management education worldwide through 
accreditation and thought leadership" (AACSB 
International, http://www.aacsb.edu/). In order to 
retain accreditation, a school must demonstrate 
that it has an assessment process in place and that 
it uses the data collected to improve its curriculum 
and student learning. The AACSB requires that 
schools of business measure student learning 
based on the school's stated mission and objectives. 
Therefore, each business school can develop its own 
assessment process and define its own learning 
expectations in the form of cross-disciplinary skills 
that students will receive during their studies. 

The guidelines require each school to identify its 
learning assurance measurements for each learning 
expectation and collect and present the results of 
these measures during AACSB reviews. 

This article contrasts the undergraduate 
assessments in two similar schools of business. In 
both schools, the faculty members determined 
the skills to be assessed, but the formats of the 
rubrics and data collection procedures employed 
by the two schools were markedly different. One 
school, Seton Hall University (SHU), adopted an 
approach utilizing mostly external resources; the 
assessment data were collected outside of the 
normal coursework and mostly by nonfaculty, 
external assessors. The other school. Pace University 
(PU), embedded assessment within the coursework 
and created an internal, organic process led by 
faculty within the existing framework of the school. 
Though the learning goals for the two programs 
were similar, there were major differences in faculty 
and student involvement with the assessment 
process at the two schools. Additionally, the top- 
down approach SHU yielded more consistent data 
from student to student and from year to year 
that could be evaluated based on measureable 
benchmarks. In contrast, the bottom-up approach 
PU relied on coursework that was available, but not 
necessarily consistent across faculty members in 
any given assessment cycle, so benchmarking was 
difficult. Descriptive profiles of the two schools are 
presented first, followed by an overview of the two 
assessment models. 



Page 3 



AIR Professional File, Number 119, Top-Down Versus Bottom-Up Paradigms of Undergraduate Business School 



/^R 

Atiaonliaii iat 
InsHtutlanat Research! 



Profiles of the Two Schools 

The two schools compete in the same 
geographical market and offer similar programs. 
SHU is a private, Catholic institution in a suburban 
location 1 4 miles outside of a major city. At the 
time of the study, it had about 5,300 undergraduate 
students with most (82%) of them living on campus. 
The average class size at SHU was 25 students with 
a 14:1 student-faculty ratio. There were 860 faculty 
members, 90% of whom held terminal degrees. 
Tuition and fees were $28,1 50 at SHU, and 90% of 
students received financial aid. 

PU is a private institution with two campuses. 

One is located in the center of a major city, while 
the other is in a suburban area outside of the city, 
similar to SHU's campus. At the time of the study, 
the undergraduate enrollment was about 8,300, and 
64% of the students commuted to the campuses. The 
average class size at PU was in the mid-20s. There 
were 460 full-time faculty members, 88% of whom 
held terminal degrees. Tuition and fees were $31,357 
at PU, and over 80% of students received financial aid. 

Assessment in Response to the 
Accreditation Board 

In preparation for AACSB reaccreditation, SHU 
implemented a new undergraduate curriculum in 
1 998. In addition to coursework in the traditional 
disciplines of business (i.e., accounting, economics, 
finance, information technology, international 
business, law, management, marketing, and 
statistics), the stated objective of the new program 
was to enhance the students' competencies in five 
areas that were identified as important for one's 
professional business career. These skills were 
identified by a committee of faculty from different 
departments together with some input from alumni 
and a review of the applicable literature. The 
following areas were identified and termed "the five 
competencies": (a) communication, (b) teamwork, 

(c) critical thinking, (d) change management, and (e) 
technology. Each of the school's departments was 
asked to adopt one or two competencies to infuse 
the skills into the coursework. Initially, the Associate 
Dean for Undergraduate Studies devised most 



of the instruments and materials for assessment 
and supervised the assessment's implementation. 
Later on, the Dean appointed a Director of 
Undergraduate Assessment, who reported to the 
Associate Dean of Undergraduate Studies. 

PU also began its assessment effort while facing 
AACSB reaccreditation, but it implemented the 
process differently than SHU. In February 2001 , 
in preparation for the AACSB review in 2006, the 
Dean of PU established a Select Committee for 
Learning Assessment. The committee's task was to 
evaluate what PU was doing "to assess its programs 
and student learning and to recommend a more 
comprehensive assessment program that will 
lead to an assessment culture at the school." The 
Dean established a special committee outside 
of the standing faculty committees because of 
the amount of work that the task required. The 
committee consisted of faculty from each academic 
department and administrators, including the 
Associate Dean and Director of Undergraduate 
Programs and the Associate Dean and Director of 
Graduate Programs. Meeting twice a month, the 
committee evaluated the assessment activities 
already established at the school (such as alumni 
and employer surveys) and conducted research on 
the assessment plans of PU's benchmark schools 
and other schools that were perceived as leaders in 
the area of assessment. 

The Select Committee recommended that 
the school hire an Associate Dean of Planning, 
Research, and Assessment to handle all assessment 
plans and procedures at the business school. 

The committee also determined that a set of 
KSAs (knowledge, skills, and abilities), which was 
previously identified by the faculty, should be the 
focus of the assessment. This set included 39 KSAs 
based on input from business leaders, faculty, and 
alumni and created for a prior reaccreditation cycle. 
Because 39 KSAs would be difficult to measure, 
the Select Committee combined similar KSAs to 
create six general categories. Those categories 
were (a) communication; (b) critical thinking, 
problem solving, and decision making; (c) financial 
and accounting applications; (d) quantitative 
applications; (e) interpersonal interaction; and (f) 
active citizenship in a global environment. 
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Assessment Tools and Measures 

Both schools designed tools and measures 
for assessing student learning. Undergraduate 
business education is very similar across 
schools and programs because of accreditation 
requirements. Therefore, the two schools had 
similar expectations of students' abilities upon 
graduation. The foci of the two assessment 
processes and the corresponding measures and 
tools are presented in this section. 

Competencies versus KSAs. The sets of skills 
that students should have upon graduation 
employed by the two schools as the foundations 
of their assessment processes are described in 
detail in Table 1 . The two sets of skills (termed 
Competencies by SHU and KSAs by PU) were similar 
regarding the communication, teamwork, and 
critical thinking skills. However, SHU did not address 
the assessment of quantitative skills explicitly, 
although the case questions used in the assessment 
required students to use such skills. On the other 
hand, because the finance department at PU was an 
early adopter of assessment activities, it had more 
influence on the design of the school's assessment 
model than other departments. Therefore, the KSAs 
included two separate quantitative skills: financial 
and accounting applications and quantitative 
applications. Departmental influences were also 
evident in SHU's assessment program where 
two competencies — teamwork and managing 
change — reflected the more active role of the 
management department (as compared with the 
other departments) in the design of the assessment 
model. 

SHU was an early adopter of pedagogical 
technology and had leased PCs to its students and 
provided online connections across the campus 
before many other universities. Technological skills 
were an important component of the school's 
curriculum and were included in the assessment 
model. PU did not include technology in its KSAs 
because it relied on assessment conducted by its 
liberal arts program to cover technological skills and 
also because it did not offer a major in Management 
Information Systems. 



Both schools developed a set of learning goals 
that were measureable and reflected student skills 
in the two programs. The major difference in the 
top-down versus bottom-up approaches was in 
measurement and implementation of assessment 
processes as discussed in the following sections. 

Assessment materials. In SHU, the primary 
assessment instruments were team presentations 
of answers to cases' questions, determined by 
the Director of Assessment, before panels of 
nonfaculty business people. Randomly, students 
were assigned to teams during their sophomore 
or senior year (i.e., each student participated in 
the assessment only once, either as a sophomore 
or senior). The cases selected for the sophomores 
were less comprehensive than the cases selected 
for the seniors and focused on law and economics 
(the areas that sophomore business students had 
already covered in their coursework), whereas 
the seniors' cases addressed accounting, finance, 
management, marketing, and statistics. The teams 
were scheduled for participation in assessment 
panels held about six to eight weeks after the start 
of the fall and spring terms. A few weeks before, the 
teams received a case and corresponding questions 
and also attended preparatory workshops held by 
the Director of Assessment. During the panels, the 
teams presented the cases to a group of outside 
assessors. The assessors were alumni and employed 
or retired business professionals. The assessors (two 
or three per team) received criteria for evaluating 
the team's performance on the five competencies, 
a scale for ranking each team on each competency, 
and a blank area for comments. The Director of 
Assessment then determined whether a team had 
passed or failed the assessment. The assessment 
panels were held outside the students' normal 
coursework, on a Friday evening and the following 
Saturday, so that they did not conflict with the 
students' attendance of regularly scheduled courses. 
Initially, faculty were not invited to observe the 
panels, but were later on (about six years after the 
first assessment panels were held) encouraged to 
do so. However, few faculty members have attended 
the panels. 
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Table 1 

Definitions of Competencies and KSAs 



SHU Competencies 


PU KSAs (knowledge, skills and abilities) 


Communication: The ability to communicate effectively, 
in writing and orally, with people of diverse business and 
professional backgrounds, both within and outside the 
organization. 


Communication: Given a realistic business problem, students will write a 
report and effectively deliver a presentation that explains a situation to a 
business audience, analyzes its implications, and makes recommendations 
for action. 


Teamwork: The ability to work with others as part of a team. 

The focus here is on developing an understanding of the 
principles of teamwork: working toward a common goal, sharing 
leadership responsibilities and authority, sharing power, sharing 
information to utilize resources fully, loyalty and collegiality, 
and building and maintaining personal and professional 
relationships. 


Interpersonal Interaction: Students will demonstrate sensitivity to others 
in two-person and small-group situations by representing their own 
interests clearly; listening to and drawing out others; acknowledging 
conflict and using it functionally as a source of creativity and energy; and 
accepting mutual accountability for decisions and actions, attempting 
to exert positive leadership when appropriate, and delivering on their 
commitments. Students will also show evidence of the application of legal 
and ethical principles in solving business problems with a concern for the 
impact of their decisions and actions on society. 


Critical Thinkina: This competency refers to the ability to identify 
relevant issues or variables, analyze their interrelationships, and 
conceptualize solutions for specific problems. Inherent to this 
competency are proficient thinking in abstract terms, being 
able to see the "big picture," and understanding how the various 
parts of an organization or an idea fit and function together. 


Critical Thinkina, Problem Solvina and Decision Makina: Given a realistic 
business situation, graduates will formulate a problem statement or 
statements, analyze the problematic issues, develop alternative courses 
of action, gather relevant data, evaluate alternatives, decide on an action, 
and set a plan for evaluating the consequences of their action. 


Chanae ManaaementThe ability to respond to, and/or 
initiate change. The management of change competency is 
focused on the students' skills in four areas: managing change 
within oneself as one matures and grows, managing change 
within organizations, understanding and responding to the 
dynamic domestic and global business environment, and 
solving creatively business problems generated by a changing 
environment. 


Active Citizenship in a Global Environment: Students will show an 
awareness of the global interdependency of business activities and an 
understanding of world markets from financial, political, economic, and 
social perspectives in their analysis and resolution of business problems. 
In their interpersonal and business-related interactions, students will 
demonstrate that they value individual and cultural differences and 
respect learning outside the focus of their business program (including, 
but not limited to, the arts and sciences).. 


Technology: Expertise in utilizina technoloavto improve 
productivity. The focus is to develop both computer literacy 
(that is, how to use computers) and information literacy. 
Specifically, knowing how to find and gather relevant data 
from various sources, organize, summarize and analyze them, 
and create meaningful and effective information for making 
business decisions. 


Not measured 


Measured via individual student assignments (see Table 2). 


Financial & Accountina Applications: Students understand financial 
statements, calculate and explain financial ratios, explain changes in 
interest rates, use time value of money concepts, and relate financial and 
accounting management techniques to the value of the firm. Additionally, 
students will understand cost allocation, earnings per share analysis, profit 
margins, expense evaluation, accounting systems and financial strategies. 


Not measured 


Quantitative Applications: Students will be able, at an introductory level, 
to solve quantitative problems and be able to draw inferences from 
mathematical solutions for business decision making. 



In addition to the team presentation, each 
student, individually, completed two assignments. 
One assignment involved entering public financial 
data (available online) into a spreadsheet program 
(Excel) and was designed to gauge the student's 
financial accounting and information technology 
skills. The other task involved writing a short essay 



and was designed to ensure that the student had 
basic writing skills. Initially, the Director graded 
these assignments. Subsequently, grading the first 
assignment was delegated to the business faculty, 
and grading the second assignment was delegated 
to faculty from the English department. The 
students also responded to questions gauging their 
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satisfaction with the assessment process and with 
their teams. 

Students had to successfully pass both the team 
component and the two individual assignments. 
Students who failed any of the components were 
given another chance to pass them. The students 
did not receive credit for the assessment work, 
but a "completed assessment" notation on one's 
transcript was required for graduation. In addition, 
students received extensive feedback regarding 
the teams' presentations in the form of a summary 
of the assessors' comments. Students preparing 
for future assessment panels were provided with 
portions of insightful assessors' comments from the 
past. 

In PU, the materials used to evaluate students' 
proficiencies in the six KSAs were cases and a 
business simulation used in the capstone business 
course. Faculty who taught this course selected 
the materials and, with the Assessment Director, 
determined which portions of these materials 
should be used in the assessment. The students 
participated in the assessment passively. They 
received grades for the capstone course, but no 
separate feedback regarding their command of the 
KSAs. 

The materials designed by the Director 
of Assessment in SHU clearly measured the 
competencies across all students and were 
consistent in form. In contrast, since PU relied 
on course materials, not all of the skills could be 
assessed across the same student population 
using the same materials. For example, a professor 
in one section of the capstone management 
course might assign a different case than another 
professor in another section leading to inconsistent 
measurement of the skills. PU was able to 
benchmark using the simulation game that was 
required content in all capstone management 
courses, however, that only covered interpersonal 
skills and quantitative skills. 

Assessment rubrics. Table 2 contains a 
summary of the materials and the rubrics used by 
the two schools. SHU relied primarily on ratings 
of student teams by the assessors who received 
written criteria to rate students' mastery of that 



given competency on a 5-point scale; the assessors 
were also asked to provide written comments 
explaining their ratings. Some guidelines instructed 
the assessors to use observations (e.g., for rating 
the teams on oral communication and teamwork), 
while others asked them to use provided answers 
to case questions as benchmarks for evaluating 
the teams' answers to those questions (e.g., for 
evaluating the teams on critical thinking and 
change management). On the other hand, PU 
used both ratings of individual students' works by 
faculty as well as more objective measures, such as 
multiple-choice questions and team performance 
in a quantitative, computer-based simulation for 
which normative performance benchmarks existed 
(in the form of data collected from many schools 
using the same simulation in their undergraduate 
capstone business courses). 

Advantages and Disadvantages of the Two 
Assessment Models 

The advantages and shortcomings of each 
assessment model are discussed next. 

Qualitative dimensions. As shown in Table 
2, the assessors in SHU used primarily a 5-point 
rating scale and personal observations to evaluate 
an entire team on the overall mastery of each 
competency. Two skills — written communication 
and spreadsheet/accounting skills — were also 
assessed through individual student assignments. 
On the other hand, in PU, faculty members served 
as assessors and were provided with specific 
dimensions on which to evaluate complex skills 
such as critical thinking, problem solving, and 
decision making. PU also used quizzes, where 
students individually responded to multiple- 
choice questions gauging their quantitative and 
accounting skills. 

Since the assessors in SHU were nonfaculty 
personnel from outside the university, who were 
unfamiliar with the intricacies of abstract skills such 
as critical thinking and managing change (although 
the definitions of such skills were provided to 
them), using closed-ended rating scales was the 
most practical solution. In addition, personal 
observations of team presentations were deemed 
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appropriate forjudging skills that can be visually 
observed such as teamwork, oral communication, 
and the use of presentation technology. In PU, 
the faculty assessors received specific dimensions 
for the faculty assessors to use in their write-ups 



assessing individual students on, among others, 
written communication, critical thinking, and 
problem solving. Providing specific dimensions for 
the write-ups of the assessments proved to be a 
somewhat more robust rubric than simply asking 



Table 2 

Rubrics of Competencies and KSAs 



Competency or KSA 


SHU 


PU 


Communication 


Observation of team presentations (enthusiasm, 
ability to maintain interest, confidence, clarity, 
and timeline). Individually completed one-page 
written essays. Faculty members from the English 
Department grade the essays. 


Cases embedded in the undergraduate capstone 
course. Students write a case analysis and give an 
oral presentation. Select faculty members evaluate 
the work along the criteria of content, organization, 
and grammar. 


Critical Thinking, Problem 
Solving & Decision Making 


Case questions that require: identifying key issues, 
the reasons for them, ability to analyze financial 
data in the context of a given business situation, 
understanding the firm's operating environment 
(internal and external) and outlining strategic 
alternatives (including the merits and potential 
shortcomings of each). 


The course-embedded cases described above 
also contain situations that require strategy 
development and implementation; planning; and 
financial and human control elements. Faculty 
members evaluate the work along the criteria of 
identifying the problem(s), analyzing the salient 
perspectives, utilizing key assumptions, critically 
examining evidence, and recommending action. 


Teamwork / Interpersonal 
Interaction 


Observation of team presentation and subsequent 
questions to team members. Criteria: selecting 
a leader, dividing the work, clearly defined 
tasks, positive group dynamics, motivation of 
team members, opportunity of each member to 
contribute, rewards for members who excelled, and 
consequences for members who did not contribute. 


Team exercises and/or cases embedded in the 
undergraduate capstone course. As part of the 
GLO-BUS 1 Management Simulation, each student 
completes a self-evaluation and evaluates his/ 
her team members on leadership, independent 
thinking, teamwork and social skills. The results 
are compared to national student sample GLO-BUS 
participants. 


Change Management/ 
Active Citizenship in a 
Global Environment 


Case questions that require students to understand 
and respond to dynamic domestic and global 
changes in the business environment. Students face 
a scenario calling for creatively solving a problem 
stemming from the changing environment. 


The same content of the cases and exercises used to 
assess Interpersonal Interaction is used. Students 
are evaluated along the criteria of identifying and 
analyzing global factors, having a world-view and 
understanding the social and ethical considerations. 
(The cases used include an ethical dimension.) 


Quantitative Applications; 
Financial & Accounting 
Applications 


Individually completes assignments in which 
students enter public financial data into a 
spreadsheet and compute several financial ratios. 
Faculty members from the business school grade 
the assignments. 


As part of the GLO-BUS simulation, students make 
decisions for a fictitious firm in the digital camera 
market. The decisions are based on quantitative 
data provided to the students. In addition, students 
complete quizzes requiring them to make financial 
decisions. The business disciplines represented 
include finance, accounting, and operations 
management. 


Technology 


Teams are required to use presentation technology. 
The assessors rate quality of the members' ability 
to use the software, quality of the visuals, special 
features that enhance the audience's interest, 
and the congruency between the visuals and the 
material presented orally. 


Not measured. 



1 GLO-BUS is a computer-based exercise where class members are divided into teams and assigned to run a company that manufactures a 
digital camera in head-to-head competition against other companies making the same product run by other class members. It is designed to 
simulate a real business environment. All teams begin the task in the same financial position. At each interval, they have to make decisions based 
on the results of their previous decisions and the standing of other companies — data that they periodically receive. Students also complete 
questionnaires rating themselves and the work of fellow members. 
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assessors for comments explaining a rating on a 
scale (the method used in SHU) because the latter 
approach often resulted in general comments that 
did not address the complexities of more intricate 
and less observable skills, such as critical thinking. 
Furthermore, all of the rubrics used at SHU were 
developed internally and cannot be compared 
against external data. On the other hand, some 
of the rubrics used in PU are used in many other 
business schools and the results could be judged 
against benchmarks solidly derived from large 
samples of students from other institutions. 

Faculty involvement. SHU has deployed its 
assessment model longer than PU, although the 
faculty of PU had identified the skills that needed 
to be evaluated even earlier (the result was the 39 
original KSAs mentioned earlier). However, in the 
Fall 2008 semester, 1 5 faculty members at PU were 
directly involved with undergraduate assessment 
activities (by providing data or analyzing data). 

Eight additional faculty members were directly 
involved in providing graduate assessment data 
and were therefore familiar with the processes. 

The Director of Assessment was a faculty member 
who continued teaching while supervising the 
assessment. The Select Committee for Learning 
Assessment consisted of an additional five faculty 
members who planned assessment activities. As 
the PU program grows, more faculty members will 
be involved in assessment, building familiarity with 
the assessment process and working to improve 
student skills. Additionally, the stipends paid to 
faculty who serve as evaluators help engender more 
positive feelings about the process. 

In SHU, five faculty members coordinated the 
definition of the five competencies, but once the 
implementation of the program began, outside 
assessors evaluated student skills, resulting in less 
faculty involvement. Initially, faculty members were 
designated as the directors of the assessment. 
However, for the last several years, the assessment 
has been directed by a nonfaculty person bearing 
the title of Associate Dean for Assessment and 
Special Projects. In addition, initially, faculty 
members were neither encouraged to attend the 
assessment panels nor involved in the selection or 



grading of the assessment materials. After several 
years of this practice, about two years ago, faculty 
members became more involved grading individual 
assignments and also were encouraged to attend 
the assessment panels. Nevertheless, the evaluation 
of the team presentations, which are the core of 
the assessment, remained the responsibility of the 
outside assessors and the nonfaculty director. 

Continuous improvement. In SHU, when the 
competencies were originally developed, each 
department was asked to adopt a competency and, 
in addition, the critical thinking skill. The objective 
was to integrate the teaching of these skills into 
the coursework, but no formal guidelines were 
put in place. In recognition of the need for a more 
aggressive approach regarding such integration, 
the school's Academic Standards Committee was 
asked to re-examine the five competencies and 
recommend more robust integration of the results 
of the assessment into the faculty's teaching. 

In order to use the assessment's findings 
to improve its undergraduate program, PU 
incorporated this activity into its faculty 
performance review process, which is conducted 
annually and determines merit pay. The Associate 
Dean and Director of Undergraduate Programs 
created a set of core course guidelines for faculty 
to follow that included a set of learning goals 
reflecting the KSAs for each course. Then, in 
their annual self-reviews, as part of the teaching 
component, faculty members were asked to 
demonstrate how they taught the applicable KSAs 
in their courses. As this process continues, faculty 
will increasingly incorporate these learning goals 
into their courses so that students can master the 
KSAs by the time they graduate. 

Financial resources and added value. SHU 
invested considerably more resources than PU 
in dedicated personnel and in developing an 
assessment structure. SHU used its assessment 
to gain publicity (e.g., exposure in the AACSB's 
publications and conferences) and also in its 
promotional efforts (e.g., open houses), where 
the five competencies and the assessment have 
been presented as a unique, value-added aspect 
of the education provided by the school. The 
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opportunity for students to participate in a business 
presentation in front of business people — who 
had not met the students before — and to receive 
feedback from them was presented as a valuable 
learning opportunity focused on the pragmatics of 
business. Mandating students, randomly assigned 
into teams and outside of the normal coursework, 
to design a team presentation based on a case 
also increased the students' engagement in the 
school's extracurricular activities, such as student 
clubs, where students could meet peers and receive 



guidance from those who had already completed 
the assessment. It must also be noted that some 
students in SHU resented the assessment because 
it consisted of an additional assignment outside of 
their coursework for which they received no credit. 
On the other hand, in PU, students were unaware of 
the assessment because the data were generated 
from their regular coursework and evaluated after 
the semester ended. 

Table 3 includes an overview and summary of 
the key features of the two assessment models. 



Table 3 

Key Features of the Two Assessment Models 





Characteristic 


Top Down: Seton Hall University 


Bottom Up: Pace University 


1 


Private 


Yes 


Yes 


2 


Urban 


Yes 


Yes 


3 


Residential students 


82% 


36% 


4 


Size 


5,300 undergraduates 


8,300 undergraduates 


5 


Financial aid recipients 


90% 


80% 


6 


Faculty terminal degrees 


90% 


88% 


7 


Assessment leadership 


Director, then Associate Dean 


Associate Dean 


8 


Identified competencies/skills 


Yes: 5 


Yes: 6 


9 


Assessment goals/materials 


Homegrown 


Homegrown & externally developed 
business simulation 


10 


Assessment location 


Outside coursework 


Within coursework 


11 


Test frequency 


2 nd and 4 th year 


Capstone class 


12 


Scoring 


Subjective (ratings) 


Subjective & objective 


13 


Raters 


Mostly panels of outside judges, 
faculty grade some assignments 


Faculty 


14 


Feedback provided to students 


Yes 


No 


15 


Assessment results integrated into 
curriculum 


Departments adopt selected 
competencies. Academic Standards 
Committee is charged with creating 
an operational framework. 


Learning objectives set for specific 
courses 


16 


Assessment reflected in faculty 
performance review 


No 


Yes 


17 


Visibility 


Yes (used in promotional materials) 


No 


18 


Costs 


Director of Undergraduate Assessment 
($20,000 per year); Pay to outside 
assessors $250 per team observation 
by 2 or 3 assessors per team (per semester 
$250 x 50 = $1 2,500); Materials duplication 
and reprint costs (per semester) $1 ,000. 


Director of Assessment - 2 course 
releases per semester ($16,000); Faculty 
Assessors - 8 @ $500 ($4,000); Incentives 
to faculty to provide materials - 
7 @$300 ($2,100). 
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Conclusions and Recommendations for 

Assessment 

Two methods of assessment are detailed here, 
each with its own advantages and shortcomings. 

We advise schools to employ a hybrid technique 
combining the most effective aspects of each 
approach, and this section includes operational 
suggestions for a hybrid system. 

Management and personnel. The development 
and implementation of the assessment models 
described in this article stemmed from the AACSB 
accreditation processes. In each case, the student 
skills assessed were identified by the faculty. Then, 
in one school, when implemented, the process 
used mostly external personnel and was ultimately 
directed by a nonfaculty Associate Dean. In the 
other school, faculty participated in collecting the 
assessment data under the direction of a tenured 
faculty member. The more effective structure for 
assessment is one managed by an Assessment 
Director who is part of the faculty and teaches 
undergraduate business courses regularly, rather 
than a nonfaculty administrator. An assessment 
process managed by a faculty member facilitates 
both the faculty involvement in selecting the 
assessment materials as well as the use of the 
assessment's results in the improvement of the 
curriculum and faculty pedagogy. 

However, engaging nonfaculty, business 
professionals as assessors provides the students 
with a unique learning experience because they 
are evaluated by "real world" business standards 
and by persons unbiased by the students' work in 
regular courses. We recommend a model where 
the assessment data are evaluated by both faculty 
and nonfaculty business professionals because such 
a structure is likely to provide fresh and additional 
insights regarding measuring key business skills. 

When using the bottom-up approach to 
assessment in the effort to involve faculty, it is very 
important to compensate faculty for assessment 
activities. Asking faculty to assess student work 
outside of the normal course structure without 
compensation would result in negative feelings 
about the assessment process and its value to 
the school. Showing faculty that the process is 



meaningful is important, particularly in the early 
stages of assessment before the processes have 
been incorporated into the regular operations of 
the school. 

Materials and rubrics. In one school, an 
Associate Dean developed the assessment materials 
(consisting mostly of a cross-disciplinary case) and 
the corresponding rubrics, and outside personnel 
were trained in applying these measures. (The 
difficulties related to this training were discussed 
earlier.) In the other school, the faculty developed 
the rubrics and using the capstone business course 
as an assessment tool, provided an opportunity 
to compare some assessment data to national 
benchmarks, indicating which areas needed 
improvement. 

On the other hand, using materials that are 
customized and not used in regular courses is more 
valuable in assessing unique skills stemming from 
a given school's mission statement (e.g., ethics 
and social responsibility). Such materials also 
serve to assess all students in a given cycle in the 
same manner and make it possible to compare 
data over time. The best approach is combining 
data from core and upper-level courses with a 
cross-disciplinary case that can be administered 
in a capstone course, and the rubrics employed 
should include measures that enable examining 
the data collected against national benchmarks. 
Relying on course materials is problematic in 
terms of validity since course content differs across 
sections of the course. Therefore, schools should 
consider developing cases and other materials 
and encourage faculty to use them when their 
sections are chosen for assessment. The results 
using the same materials in all sections would allow 
for comparison across sections and even from 
assessment cycle to assessment cycle. 

Student participation. In one school, the 
assessment process was highly visible and the 
students actively participated in it and received 
feedback about their performance. In the other 
school, the students were unaware that they were 
being assessed since the assessment was designed 
to provide data to the administration without 
student involvement. Advocating the second 
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approach, one can claim that an unobtrusive 
measure of the skills senior students presumably 
have after completing most of their coursework 
is more credible than an obtrusive measure 
where students prepare for assessment panels 
and complete assessment assignments. However, 
the highly visible assessment at SHU, including 
the presentations in front of business people, 
provides the participants with a distinctive and 
worthy lesson, in addition to involving alumni and 
seasoned businesspeople in the life of the school. 
Thus, if a school has adequate resources, creating 
a highly visible assessment structure designed to 
provide an additional pedagogical experience to 
the students is advisable. 

Faculty engagement and continuous 
improvement. At the present time, both schools 
are actively engaged in assessment activities with 
an eye toward continuous improvement. One 
school has started with a model relying strictly 
on outside assessors, moved towards a paradigm 
stressing more faculty participation, and is still 
grappling with methodically integrating the 
assessment's findings into the courses taught. The 
model employed by the other school, where the 
assessment data are collected with the assistance 
of the faculty and where the faculty's performance 
review reflects the actual use of the assessment 
findings, actively engages the faculty sooner so that 
continuous improvement can be more seamless. 
Therefore, incorporating the assessment's results 
as one of the criteria used to evaluate a faculty 
member's performance (and to determine merit 
pay where such pay is available) is strongly 
recommended. 

It must be noted that the challenges faced by 
the two models described here are not unique. In 
a recent compilation of articles about assessment 
published by the Association for Institutional 
Research (AIR), one article addressed faculty 
claims that assessment contradicts academic 
freedom (Williams, 2005). Another paper in the 
volume characterized typical faculty responses to 
assessment as being one of the following:"! can't," 
"I've already done that, ""I shouldn't have to," and 
"I won't" (Martell 2005, p. 210). Presumably, such 



responses reflect faculty reluctance to gather 
assessment data and also modify the contents 
of their courses in a way that addresses the 
inadequacies identified by the assessment. A review 
of the papers published in AIR's volume reveals 
that many institutions with sound (and costly) 
assessment structures have yet to find a smooth 
way to integrate the results of the process into the 
contents of the courses taught. 

In addition to engaging faculty in the 
assessment itself (with some financial incentives) 
and incorporating the use of assessment results 
into the faculty's performance review, creating an 
operational framework that guides the faculty in 
implementing continuous improvement is needed. 
For example, following the identification of the skills 
to be assessed and the rubrics measuring students' 
mastery of these skills, one assessment program 
created a detailed table matching the skills with 
stated learning outcomes spread along four years of 
undergraduate education (Popper 2005, pp. 1 8-1 9). 

In summary, the following quote from one of 
the papers in AIR's volume summarizes the state 
of assessment in many schools: "While we have 
come a long way in a year, our assessment is neither 
complete nor perfect — it is a work in progress" 
(Anderson-Fletcher, 2005, p. 75). As schools come 
up for AACSB re-accreditation, assessment will 
become embedded in the professional mindset of 
business faculty and will also be reflected in their 
teaching. 
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